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Since  1988,  the  Department  of  Defense  (dod)  has  been  grappling  with  how 
to  provide  simulations  that  realistically  portray  joint  warfare  operations 
for  training.  To  help  meet  this  training  need,  the  Defense  Advanced 
Research  Project  Agency  developed  the  Aggregate  Level  Simulation 
Protocol  (alsp).  The  alsp  technique  provides  a  means  for  multiple  service 
and  agency  models  to  communicate  with  each  other.  Although  alsp  is  a 
technological  achievement,  the  existing  warfare  cjqjabihties  of  the 
individual  models  is  hmited,  and  therefore,  the  problem  of  providing  a 
valid  joint  training  environment  remains.  Overcoming  these  limitations 
would  require  extensive  improvements  to  the  alsp  Confederation.^ 

Because  of  these  hmitations  and  the  cost  and  complexity  of  the  alsp 
Confederation,  the  mihtary  services;  the  Joint  Staff;  and  the  Office  of  the 
Secretary  of  Defense’s  Director,  Defense  Research  and  Engineering, 
initiated  the  development  of  a  comprehensive  system  that  would  t^e  the 
place  of  the  alsp  Confederation.  The  new  system — ^the  Joint  Simulation 
System  (jsims) — ^is  scheduled  to  reach  initial  operational  capabihty  by  1999 
and  full  operational  capability  by  2003. 

Until  JSIMS  is  operational,  dod  plans  to  continue  making  improvements  to 
the  ALSP  Confederation.  Because  dod  has  repeatedly  acknowledged  a  need 
to  improve  joint  training  through  cost-effective  simulations,  we  initiated  a 


’For  the  purposes  of  this  report,  references  to  the  ALSP  Confederation  include  the  conununication  link 
and  the  group  of  models  operating  through  the  ALSP  interface  for  training. 


Page  1 


GAO/NSIAD-96-44  Military  Training 


B-262250 


review  to  determine  whether  (1)  dod  is  progressing  with  its  development 
of  JSiMS  and  (2)  dod’s  decisions  to  improve  the  alsp  Confederation  are 
cost-effective.  We  conducted  this  review  under  our  basic  legislative 
responsibilities  and  are  addressing  this  report  to  you  because  we  beheve  it 
will  be  of  interest  to  your  committees. 


Results  in  Brief 


The  JSIMS  initiative  has  not  progressed  beyond  the  conceptual  stage  since  a 
memorandum  of  agreement  was  signed  in  June  1994.  The  services  and  the 
Joint  Staff  have  disagreed  about  the  definition  of  jsims  and  a  plan  of  action. 
The  Under  Secretary  for  Acquisition  and  Technology  has  decided  to  let  the 
services  and  the  Joint  Staff  resolve  their  differences  over  time  rather  than 
take  conclusive  actions.  Further,  the  estimated  $416  milhon  in  funding 
needed  to  develop  jsims  wUl  be  dependent  upon  agreement  by  multiple 
sources — ^the  services  and  other  agencies. 

DOD  is  uncertain  about  how  much  it  will  spend  to  improve  the  alsp 
Confederation  before  replacing  it  with  jsims.  This  uncertainty  raises 
questions  as  to  whether  dod  is  making  cost-effective  decisions.  We 
identified  about  $40  milhon  that  dod  plans  to  spend  through  fiscal  year 
1999  for  ALSP  Confederation  improvements;  however,  additional  monies 
may  be  spent,  depending  upon  the  individual  service  and  Commander  in 
Chief  (CINC)  priorities. 

Many  of  the  improvements  to  the  alsp  Confederation  are  likely  to  be 
completed  at  about  the  same  time  that  it  is  scheduled  to  be  replaced  by 
JSIMS.  The  longer  it  takes  to  make  jsims  operational,  the  more  money  dod  is 
likely  to  spend  on  a  system  that  wih  ultimately  be  discarded.  Moreover, 
because  the  services  are  already  developing  their  next  generation  of 
training  models  without  a  clear  vision  of  their  relationship  to  jsims,  the 
services  could  duphcate  costs  by  unnecessarily  budding  capabilities  that 
wUl  already  be  included  in  jsims. 


Background 


Because  the  models  that  comprise  the  alsp  Confederation  were  buUt  in  the 
early  1980s  to  meet  service-specific  needs,  they  lack  the  abUity  to  simulate 
many  aspects  of  joint  warfare,  including  operations  other  than  war, 
strategic  mobility,  space,  intelligence,  and  logistics  capabUities.  These 
models  also  lack  the  capabUity  to  represent  many  combat  interactions, 
such  as  ground  to  ship.  Because  of  the  existing  shortfaUs  of  the  services’ 
individual  models,  the  alsp  Confederation  can  only  fuUy  support  2  of  26 
identified  ciNC  and  service  joint  training  requirements.  This  also  may  be 


Page  2 


GAO/NSIAD-96-44  Military  Training 


B-262250 


the  result  of  the  fact  that  requirements  for  apphcation  of  the  alsp 
technology  were  solicited  from  the  ciNCs  and  services  only  after 
development  of  the  technology. 

The  services  have  long  recognized  the  technical  and  training  shortfalls  of 
their  respective  models  for  accurately  portraying  joint  operations.  The 
Army’s  Corps  Battle  Simulation  is  a  ground  maneuver  training  simulation 
used  in  exercises  for  commanders  and  battle  staffs.  The  Army  model  lacks 
the  capabOity  to  simulate  weather  information,  the  terrain  of  the 
battlefield,  and  ground-to-ground  combat  interactions  with  the  Marine 
Corps  model.  The  Air  Force’s  Air  Warfare  Simulation  used  to  support  air 
operations  has  limited  capability  to  simulate  electronic  warfare, 
reconnaissance  and  surveillance  play,  and  space  capabilities.  'The  Navy’s 
sanctioned  training  model,  the  Enhanced  Naval  Wargaming  System, 
operates  on  a  hardware  system  that  cannot  interface  with  alsp.  The  Navy 
has  been  modifying  this  model  for  acceptance  into  the  confederation  since 

1993.  Navy  officials  were  unable  to  elaborate  on  the  joint  training  benefits 
that  would  be  achieved  from  these  modifications.  According  to  service 
modehng  and  simulation  officials  and  after-action  reports,  the  Research, 
Evaluation,  and  Systems  Analysis  Simulation — a  naval  analytical 
model — ^has  been  used  successfully  in  the  current  alsp  Confederation.  In 

1994,  the  Marine  Coips  introduced  a  new  amphibious  operations 
simulation,  the  Marine  Air  Ground  Task  Force  Tactical  Warfare 
Simulation,  into  the  alsp  Confederation. 

The  Office  of  the  Secretary  of  Defense  created  the  Defense  Modeling  and 
Simulation  Office  to  serve  as  the  focal  point  for  modeling  and  simulation 
under  the  Director,  Defense  Research  and  Engineering.  'The  dod  Executive 
Council  for  Modehng  and  Simulation,  chaired  by  the  Director,  Defense 
Research  and  Engineering,  advises  and  assists  the  Under  Secretary  of 
Defense  for  Acquisition  and  Technology  in  modehng  and  acquisition 
decisions.  The  jsims  program  is  a  jointly  managed  dod  program  with  the 
Air  Force  providing  acquisition  oversight.  The  jsims  Joint  Program  Office, 
under  the  Air  Force  Program  Executive  Officer  for  Combat  Support 
Systems,  has  been  designated  as  an  acquisition  activity  for  jsims.  'The 
Army’s  Simulation,  Training  and  Instrumentation  Command  is  the 
executive  agent  for  the  day-to-day  management  of  the  alsp  Confederation. 


JSIMS  Proceeds  at  a 
Deliberate  Pace 


The  development  of  jsims  is  already  a  year  behind  schedule  and  a  clear, 
consistent  definition  of  jsims  is  sthl  evolving.  According  to  the  June  1994 
joint  memorandum  of  agreement,  a  clear  definition  was  due  of  what 
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constitutes  jsims  within  4  months  of  the  signing  of  the  memorandum.  Also 
due  was  a  detailed  plan  of  action  in  the  form  of  a  jsims  Joint  Program 
Office  charter  and  jsims  master  plan  delineating  duties,  responsibilities, 
mission,  scope,  and  strategies  for  implementing  jsims.  However,  lack  of 
agreement  among  the  services  as  to  what  jsims  entails  has  delayed 
approval  of  the  charter  and  the  plan.  The  services  have  different 
interpretations  of  the  memorandum  of  agreement.  The  low  end  of 
expectations  is  a  set  of  standards  and  protocols  that  would  allow 
interoperability  for  the  services’  next  generation  of  simulations.  The  high 
end  of  expectations  is  a  “super  model”  in  which  jsims  would  describe  all  of 
the  objects,  such  as  aircraft,  for  all  of  the  services  and  determine  all 
warfare  functions. 

During  July  1995,  the  Acting  Assistant  Secretary  of  the  Air  Force  for 
Acquisition  approved  milestone  0  for  the  jsims  program,  which  authorizes 
proceeding  into  the  concept  exploration  and  definition  phase  of  the 
acquisition  cycle.  At  that  time,  the  jsims  Joint  Program  Office  stated  that 
JSIMS  would  comprise  (1)  a  core  element  of  common  functions,  such  as 
terrain  and  weather  effects  and  (2)  warfare  functions,  such  as  air,  ground, 
and  naval  combat,  and  logistics.  Common  core  development  would  be  the 
rei^onsibility  of  the  jsims  Joint  Program  Office  while  warfare  function 
development  will  be  the  responsibility  of  designated  executive  agents.  The 
executive  agents  will  develop  a  joint  representation  of  their  warfare  area 
that  would  then  be  integrated  with  the  jsims  core.  The  Army  is  the 
executive  agent  for  land  warfare,  the  Air  Force  for  air  and  space  warfare, 
and  the  Navy  for  sea  warfare.  The  Marine  Corps’  missions  will  be  included 
throughout  these  executive  agents’  warfare  representations. 

Further,  the  1994  memorandum  of  agreement  stated  that  jsims  should  also 
be  adaptable  to  other  modeling  and  simulation  applications,  such  as 
analysis  and  testing.  However,  in  February  1995,  the  Deputy  Secretary  of 
Defense  directed  the  Director,  Program  Analysis  and  Evaluation,  to  initiate 
and  lead  development  of  a  new  joint  analysis  model  called  the  Joint 
Warfare  System  (jwars).  Program  Analysis  and  Evaluation  officials 
informed  us  that  they  believed  improvements  to  dod’s  analytical  capability 
needed  to  be  made  now  and  they  could  not  afford  to  wait  for  jsims  to 
become  a  reality-  The  jsims’  focus  is  now  solely  on  providing  a  simulation 
environment  for  joint  task  force  training.  Coordination  between  the  jsims 
and  JWARS  programs  is  being  worked  out. 

Currently,  the  mayor  stumbling  block  for  jsims  is  how  to  fund  the 
$416  million  program  since  there  is  no  central  funding  line  for  the 
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program.  Some  military  service  officials  have  expressed  concerns  about 
the  piecemeal  t^proach  of  funding  jsims.  As  of  July  1995,  the  jsims’s  core 
element  was  estimated  to  cost  about  $154  milMon.  Under  the  provisions  of 
the  joint  memorandum  of  agreement,  the  Army,  the  Air  Force,  and  the 
Navy  have  each  agreed  to  provide  30  percent  of  this  cost.  The  Marine 
Corps  win  provide  10  percent  of  the  cost.  In  addition  to  the  $154  million, 
the  executive  agents  wUl  incur  additional  costs,  currently  estimated  at  a 
total  of  $262  million,  to  develop  simulations  for  their  specific  warfare 
functions. 

The  problem  with  this  approach  is  that  if  a  service  believes  that  improving 
its  own  core  competencies  has  a  higher  priority  to  fund  than  its 
responsibilities  for  jsims,  that  function  for  jsims  may  not  be  developed  in 
concert  with  the  other  required  components.  The  military  services  are 
proceeding  to  develop  the  next  generation  of  simulations  that  will  better 
address  their  specific  mission  or  core  requirements.  The  services  are  also 
responsible  for  ensuring  that  these  simulations  are  able  to  function  within 
the  jsims’  domain.  The  Army’s  program.  Warfighters’  Simulation  2000,  is 
estimated  to  cost  about  $200  million  and  be  operational  by  2000.  The  Air 
Force  is  developing  the  National  Air  and  Space  Warfare  Model  that  is 
estimated  to  cost  about  $103  million  and  be  fuUy  operational  by  2003.  The 
Navy  is  developing  an  analytical  simulation,  the  Naval  Simulation  System, 
at  an  initial  estimated  cost  between  $15  million  and  $25  million  that  could 
be  enhanced  at  an  additional  cost  of  about  $47  million  to  function  in  a 
training  capacity.  Unless  decisive  management  is  exercised,  these  service 
efforts  may  outpace  jsims’  core  development  and  require  additional 
modifications  to  operate  in  the  jsims’  domain. 

According  to  dod  officials,  several  recent  events  have  occurred  that 
demonstrate  the  jsims  program  is  moving  forward.  First,  on  July  14, 1995, 
the  Director,  Defense  Research  and  Engineering,  chaired  the  first  jsims 
Senior  Review  Board  at  which  the  members  agreed  to  provide  their  share 
of  the  jsims  Joint  Program  Office  permanent  staff.  Second,  the  Under 
Secretary  of  Defense  for  Acquisition  and  Technology  signed  a 
memorandum  on  August  8, 1995,  calling  on  dod  components  to  formally 
adopt  a  proposed  division  of  funding  and  personnel  requirements.  Third, 
the  Deputy  Secretary  of  Defense  endorsed  the  establishment  of  a  joint 
core  funding  line  with  the  services  providing  both  their  share  of  core 
funding  and  personnel  to  staff  the  jsims  Joint  Program  Office.  In  addition, 
the  Director,  Defense  Research  and  Engineering,  and  the  Joint  Staff  are  to 
provide  a  share  of  funding  for  the  jsims  core  program.  However,  we  note 
that  these  actions  have  not  been  formalized. 
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DOD’s  Expenditures 
on  the  ALSP 
Confederation  May 
Not  Be  Cost-Effective 


Concurrent  with  the  development  of  jsims,  dod  has  decided  to  make 
improvements  to  the  alsp  Confederation,  the  last  of  which  is  expected  to 
be  in  place  in  1999— at  the  same  time  that  jsims  should  reach  initial 
operational  capability.^  According  to  the  alsp  Master  Plan,  the 
improvements  are  intended  to  respond  to  the  identified  CINC  and  service 
training  requirements  and  include  additional  capabilities  such  as  strategic 
mobility  and  ground-to-ground  combat  interactions  between  models.  Even 
though  officials  from  the  Office  of  the  Secretary  of  Defense’s  Director, 
Defense  Research  and  Engineering,  the  Defense  Modeling  and  Simulation 
Office,  and  the  Army  Simulation,  Training,  and  Instrumentation  Command 
told  us  that  the  total  cost  of  these  improvements  will  not  be  significantly 
high,  none  of  these  offices  was  able  to  provide  comprehensive  cost 
estimates.  We  identified  about  $40  million  that  dod  plans  to  spend  for  alsp 
Confederation  improvements  through  fiscal  year  1999.  However,  because 
this  money  may  be  directed  toward  service-specific  improvements  rather 
than  joint  improvements,  the  cost  could  be  higher. 


As  is  the  case  with  jsims,  there  is  no  central  funding  line  for  the  alsp 
Confederation  improvements.  Consequently,  dod’s  ability  to  achieve  ail  of 
the  improvements  that  it  seeks  is  dependent  on  funding  from  the 
individual  military  services,  agencies,  or  ciNCs.  However,  to  date  the  Office 
of  the  Under  Secretary  for  Acquisition  and  Technology  has  not  provided 
the  management  to  ensure  that  aU  significant  components  of  the  alsp 
improvements  will  be  completed.  Consequently,  management  of  the 
improvements  has  been  fragmented  and  it  is  questionable  whether  the 
improvement  plan  is  cost-effective.  For  example,  the  Army  has  decided 
not  to  fund  alsp  improvements  to  its  ground  warfare  model,  which  is  a 
primary  component  of  the  alsp  Confederation.®  The  Army  is  proceeding  to 
develop  its  new  training  model,  Warfighters’  Simulation  2000.  The  Army 
has  already  awarded  contracts  for  the  new  model’s  development.  The 
impact  of  the  Army’s  decision  not  to  fund  ground  warfare  improvements 
on  other  confederation  model  improvement  efforts  or  future  training 
requirements  is  unknown. 


In  contrast,  the  Air  Force  is  spending  about  $7  million  to  consolidate  two 
versions  of  its  air  warfare  model  and  plans  to  enter  the  combined  model 
into  the  alsp  Confederation  in  1997.  The  consolidation  effort  wUl  result  in 


2JSIMS  Joint  Program  Office  officials  also  informed  us  that  JSIMS  should  be  able  to  take  over  some  of 
the  functions  of  the  ALSP  Confederation  in  the  1997-98  time  frame. 

^While  the  Army  is  not  planning  to  fund  joint  ALSP-related  improvements  to  its  ground  warfare  model, 
these  improvements  could  be  funded  by  a  CINC.  The  Army  is  funding  changes  to  this  model  for 
Armynspecific  training  purposes. 
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combining  the  best  features  of  the  two  versions,  as  well  as  preventing 
future  duplicative  efforts.  The  Navy  has  been  spending  nearly  $2  million 
annually  to  replace  its  current  confederation  model  by  fiscal  year  1997. 
Navy  officials,  however,  could  not  specify  how  the  replacement  model 
would  improve  the  confederation’s  joint  training  capability.  The  U.S. 
Transportation  Command  and  the  U.S.  Space  Command  are  each 
modifying  models  for  inclusion  into  future  confederations  that  would 
expand  the  alsp  Confederation’s  capability. 


Recommendations 


To  help  ensure  the  total  development  of  jsims,  we  recommend  that  the 
Secretary  of  Defense  establish  a  joint  funding  line  for  the  core 
development  of  jsims  and  direct  the  Secretaries  of  the  Army,  the  Navy,  and 
the  Air  Force  to  establish  funding  lines  for  their  respective  executive  agent 
JSIMS  responsibilities  regarding  warfare  function  development.  Further,  we 
recommend  that  the  Secretary  of  Defense  require  the  Under  Secretary  for 
Acquisition  and  Technology  to  assume  a  stronger  management  role  to 
resolve  simulation  issues  by 

defining  jsims  and  developing  a  definitive  plan  of  action  and 
developing  a  transition  strategy  to  phase  out  alsp  and  phase  in  jsims.  This 
strategy  should  be  based  upon  cost  estimates  associated  with  modifying, 
expanding,  and  testing  the  alsp  Confederation  to  decide  which 
improvements  to  the  alsp  Confederation  provide  benefits  that  are 
cost-effective. 


Agency  Comments 
and  Our  Evaluation 


In  written  comments  on  a  draft  of  our  report,  dod  generally  agreed  with 
our  findings  and  recommendations  (see  app.  I).  The  Department  said  that 
it  recognizes  the  shortcomings  of  today’s  joint  training  simulations  and  is 
committed  to  developing  more  cost-effective  capabilities.  In  response  to 
our  recommendations,  dod  said  that  it  has  taken  action  to  establish  a  joint 
funding  line  for  the  jsims  core  and  to  ensure  service  support  for  their 
respective  combat  representations,  dod  stated  that  a  plan  to  phase  out  alsp 
and  phase  in  jsims  will  be  developed  based  on  both  technical 
considerations  provided  by  the  Under  Secretary  of  Defense  for  Acquisition 
and  Technology  and  operational  considerations  provided  by  the  services 
and  ciNCs. 

However,  dod  did  not  agree  with  our  assessment  of  the  status  of  the  jsims 
program.  The  Department  does  not  believe  that  the  jsims  program  has  been 
stalled.  DOD  said  that  (1)  it  deliberately  established  ambitious  milestones  in 
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the  JSIMS  memorandum  of  agreement  to  serve  as  an  action  to  move  the 
project  along;  (2)  the  jsims  project  has  moved  from  a  general  consensus 
agreement,  through  stand-up  of  a  transitional  jsims  Joint  Program  Office, 
to  the  formation  of  a  permanent  Joint  Program  Office;  (3)  a  systems 
definition  for  jsims  was  developed  in  an  April  1995  meeting;  (4)  the  jsims 
Operational  Requirements  Document  is  in  final  review;  and  (5)  the 
program  officially  entered  the  Concept  Exploration  and  Definition  phase 
when  it  attained  milestone  0  status  during  July  1995. 

Our  assessment  of  the  status  of  jsims  is  based  upon  documentation 
provided  to  us  during  our  review.  The  various  management  groups 
responsible  for  development  of  jsims  have  conducted  numerous  meetings 
in  an  effort  to  bring  about  a  consensus  of  what  jsims  constitutes.  However, 
we  believe  that  jsims  has  been  stalled  at  a  fundamental  level  as  evidenced 
by  the  minimal  progress  since  the  signing  of  the  June  1994  memorandum 
of  agreement.  At  the  conclusion  of  our  review,  there  were  indications  that 
the  program  might  be  progressing.  However,  no  actions  had  been  finalized. 
A  permanent  charter  for  the  Joint  Program  Office  as  called  for  by  October 
1994  is  still  not  estabhshed.  The  jsims  Operational  Requirements  Document 
is  stiU  not  approved.  According  to  documents  presented  at  the  July  1995 
JSIMS  Senior  Review  Board  meeting,  the  estimated  cost  to  develop  jsims 
core  and  warfare  functions  is  $416  million.  The  jsims  core  without  the 
warfare  functions  will  not  achieve  dod’s  joint  training  objectives. 

Therefore,  we  believe  it  is  important  to  identify  all  development  costs. 

DOD  said  that  it  could  not  substantiate  the  $40  million  we  identified  that  the 
services  are  planning  to  spend  on  alsp  improvements,  dod  stated  that 
$6.1  miUion  is  currently  budgeted  for  alsp  core  support  through  fiscal  year 
1999.  DOD  acknowledged  that  all  other  funding  for  modifications  in  the  alsp 
models  is  provided  by  the  services  or  CINCS  but  could  not  substantiate  this 
figure.  The  $40  million  figure  was  derived  from  documents  and  discussions 
held  with  service  budget  officials  and  is  subject  to  change  depending  upon 
the  services’  priorities  for  spending. 


Scope  and 
Methodology 


To  determine  whether  dod  is  progressing  with  its  development  of  jsims,  we 
interviewed  knowledgeable  officials  from  the  Defense  Modeling  and 
Simulation  Office,  Washington,  D.C.;  the  Joint  Staff,  Washington,  D.C.;  the 
Joint  Warfighting  Center,  Fort  Monroe,  Virginia;  the  jsims  Joint  Program 
Office,  Orlando,  Florida;  and  the  services’  modehng  and  simulation 
management  offices  in  Washington,  D.C.  In  addition,  we  interviewed  the 
Director,  Defense  Research  and  Engineering,  Office  of  the  Secretary  of 
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Defense.  We  reviewed  the  draft  dod  Modeling  and  Simulation  Master  Plan; 
the  Executive  Council  for  Modeling  and  Simulation  meeting  minutes;  and 
DOD,  Joint  Staff,  and  service  modeling  and  simulation  policies.  In  addition, 
we  reviewed  related  Defense  Science  Board  and  dod  Inspector  General 
reports. 

To  determine  whether  dod’s  decisions  to  improve  the  alsp  Confederation 
are  cost-effective,  we  interviewed  modeling  and  simulation  officials  at  the 
Simulation,  Training,  and  Instrumentation  Command,  Orlando,  Florida;  the 
Warrior  Preparation  Center,  Einsiedlerhof  Air  Station,  Germany;  the  Joint 
Training  Analysis  and  Simulation  Center,  Suffolk,  Virginia;  and  the 
National  Simulation  Center,  Fort  Leavenworth,  Kansas.  We  reviewed 
numerous  documents  on  the  alsp  Confederation.  We  discussed  the  costs 
of  simulation  improvements  with  each  of  the  service  model’s  proponents. 

We  conducted  our  work  between  January  1995  and  August  1995  in 
accordance  with  generally  accepted  government  auditing  standards. 


We  are  sending  copies  of  this  report  to  the  Chairmen,  Senate  and  House 
Committees  on  Appropriations,  Senate  Committee  on  Armed  Services,  and 
House  Committee  on  National  Security;  the  Secretary  of  Defense;  the 
Deputy  Secretary  of  Defense;  the  Under  Secretary  of  Defense  for 
Acquisition  and  Technology;  the  Director,  Defense  Research  and 
Engineering;  and  the  Secretaries  of  the  Army,  the  Navy,  and  the  Air  Force. 
We  will  make  copies  available  to  others  on  request. 

Please  contact  me  at  (202)  512-5140  if  you  or  your  staff  have  any  questions 
concerning  this  report.  Major  contributors  to  this  report  were 
Charles  J.  Bonanno,  Brenda  S.  Farrell,  Raymond  G.  Bickert,  and 
Colin  L.  Chambers. 


Mark  E.  Gebicke 
Director,  Mihtary  Operations 
and  Capabilities  Issues 


Page  9 


GAO/NSIAD-96-44  Military  Training 


Appendix  I 


Comments  From  the  Department  of  Defense 


Mark  E.  Gebtcke 

Director,  Military  Operations  and  Capabilities  Issues 
National  Security  and  International  Affairs  Division 
US  General  Accounting  Office 
Washington,  DC  20548 

Dear  Mr.  Gebicke: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  General  Accounting 
Office  (GAO)  draft  report  entitled,  “Military  Training:  Cost-Effective  Development  of 
Simulations  Present  Significant  Challenges,”  dated  August  10,  1995,  and  revisions  dated 
September  14,  1995  (GAO  Code  703092),  OSD  Case  9997.  The  Department  partially 
concurs  with  the  report  and  has  implemented  or  is  in  the  process  of  implementing  most  of 
the  recommendations. 

The  draft  report  examines  the  Aggregate  Level  Simulation  Protocol  ( ALSP)  and  the 
new  Joint  Simulation  System  (JSIMS)  program,  and  it  raises  questions  regarding  their  cost- 
effectiveness  and  oversight.  The  Department  recognizes  the  shortcomings  of  today’s 
limited  set  of  joint  training  tools  and  is  committed  to  developing  more  robust  and  cost 
effective  capabilities.  The  capabilities  envisioned  for  the  new  JSIMS  program  are  a 
quantum  leap  beyond  the  current  generation  of  modeling  and  simulation  tools.  For 
example,  the  JSIMS  will  support  training  of  Joint  Task  Forces  and  combatant  commands  in 
all  phases  of  military  operations,  and  will  provide  an  interface  between  training  systems 
and  operational  command,  control,  communications,  computers,  and  intelligence  systems. 
This  unprecedented  capability  is  essential  to  maintaining  a  high  state  of  readiness  for  the 
full  spectrum  of  modem  and  future  warfare. 

It  is  important  to  note  that  the  JSIMS  was  conceived  by  the  Services  and  has 
advanced  through  their  deliberate  and  collaborative  efforts  with  the  Office  of  the  Secretary 
and  the  Joint  Staff.  Their  efforts  to  clarify  the  scope  of  the  JSIMS  and  to  coordinate 
responsibilities  during  the  early  phases  of  development  have  provided  a  firm  foundation, 
which  is  expected  to  save  time  and  money  in  subsequent  phases  of  development. 

The  Department  is  constantly  seeking  better  ways  to  improve  the  coordination  and 
management  of  modeling  and  simulation  programs.  In  this  regard,  the  organizational 
structure  created  by  Office  of  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology  has  redoubled  its  efforts  to  identify  and  integrate  modeling  and  simulation 
requirements,  develop  comprehensive  plans,  and  prioritize  investments.  These  efforts  are 
consistent  with  most  of  the  recommendations  addressed  in  the  GAO  report. 

o 
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QAO  DRAFT  REPORT — DATED  AUGUST  10,  1995 
(GAO  CODE  703092)  OSD  CASE  9997 

••MILITARY  TRAINING:  COST-EFFECTIVE  DEVELOPMENT  OF 
SIMULATIONS  PRESENT  SIGNIFICANT  CHALLENGES*' 

DEPARTMENT  OF  DEFENSE  COMMENTS 


FINDINGS 


FINDING  A:  Joint  Simulation  System  (J8IMS)  Progress 
Proceeds  At  a  Deliberate  Paco.  The  GAO  found  that  the 
development  of  the  JSIMS  is  almost  a  year  behind  schedule 
and  a  clear,  consistent  definition  of  the  JSIMS  is  still 
evolving.  The  GAO  noted  that  a  lack  of  agreement  among  the 
Services  as  to  what  the  JSIMS  entails  has  delayed  approval 
of  the  charter  and  plan.  The  GAO  pointed  out  that  the 
Services  have  different  interpretations  of  the  memorandum 
of  agreement. 

The  GAO  reported  that  in  July  1995  the  Acting  Assistant 
of  the  Air  Force  for  Acquisition  approved  Milestone  0  for 
the  JSIMS.  The  GAO  stated  that  the  JSIMS  Joint  Program 
Office  indicated  the  JSIMS  would  comprise  a  core  element  of 
common  functions,  such  as  terrain  and  weather  effects,  and 
warfare  functions,  such  as  air,  ground,  and  naval  combat, 
and  logistics.  The  GAO  noted  that  the  common  core  develop¬ 
ment  would  be  the  responsibility  of  the  JSIMS  Joint  Program 
Office,  while  warfare  function  development  will  be  the 
responsibility  of  designated  executive  agents. 

The  GAO  reported  that  the  1994  memorandum  of  agreement 
stated  that  the  JSIMS  should  also  be  adaptable  to  other 
modeling  and  simulation  applications,  such  as  analysis 
and  testing.  The  GAO  noted,  however,  that  in  February  1995, 
the  Deputy  Secretary  of  Defense  initiated  an  effort  to  be 
led  by  the  Director,  Program  Analysis  and  Evaluation  to 
develop  a  new  joint  analysis  model  called  the  Joint  Warfare 
System  (JWARS) ,  The  GAO  stated  that  Program  Analysis  and 
Evaluation  officials  advised  that  they  believed  improvements 
to  the  DoD  analytical  capability  needed  to  be  made  now  and 
they  could  not  afford  to  wait  for  the  JSIMS  to  become  a 
reality.  The  GAO  pointed  out  that  the  focus  of  the  JSIMS  is 
now  solely  on  providing  a  simulation  environment  for  joint 
task  force  training.  The  GAO  noted  that  coordination 
between  the  JSIMS  and  JWARS  programs  is  being  worked  out. 


Page  12 


GAO/NSIAD-96-44  MiUtary  Training 


Appendix  I 

Comments  From  the  Department  of  Defense 


Now  on  pp.  3-6. 


The  GAO  stated  that  the  major  stumbling  block  for 
the  JSIMS  is  how  to  fund  the  $416  million  program,  since 
there  is  no  central  funding  line  for  the  program.  The  GAO 
noted  that  some  of  the  Military  Service  officials  have 
expressed  concerns  about  the  piecemeal  approach  of  funding 
the  JSIMS,  The  GAO  reported  that  the  current  estimate  of 
the  JSIMS  core  element  is  about  $154  million.  The  GAO 
noted  that  the  Services  have  each  agreed  to  provide 
30  percent  of  the  cost  and  the  Marine  Corps  will  provide 
10  percent  of  the  cost.  The  GAO  reported  that  in  addition 
to  the  $154  million,  the  executive  agents  will  incur 
additional  costs,  currently  estimated  to  total  $262  million, 
to  develop  simulations  for  their  specific  warfare  functions. 

The  GAO  concluded  the  problem  with  this  approach  is 
that  if  a  Service  believes  that  improving  its  own  core 
competencies  has  a  higher  priority  to  fund  than  its  respon¬ 
sibilities  for  the  JSIMS,  that  function  for  the  JSIMS  may 
not  be  developed  in  concert  with  the  other  required 
components.  The  GAO  noted  that  the  Services  are  proceeding 
to  develop  the  next  generation  of  simulations  that  will 
better  address  their  specific  mission  or  core  requirements. 
The  Services  are  also  responsible  for  ensuring  that  these 
simulations  are  able  to  function  within  the  JSIMS ^s  domain. 
The  GAO  stated  that  unless  decisive  management  is  exercised, 
these  Service  efforts  may  outpace  the  JSIMS  core  development 
and  require  additional  modifications  to  operate  in  the  JSIMS 
domain. 

The  GAO  reported  that  several  recent  events  have 
occurred  that  demonstrate  the  JSIMS  program  is  moving 
forward.  The  GAO  noted,  however,  that  these  actions 
have  not  been  formalized.  {pp.  2-3,  pp.  4-7/GAO  Draft 
Report) 


POD  RESPONSE:  Partially  concur.  Milestones  established  in 
the  JSIMS  Memorandum  of  Agreement  (MOA)  were  deliberately 
ambitious  and  intended  as  an  action  to  move  the  JSIMS 
project  along.  The  JSIMS  MOA  was  signed  on  July  15,  1994, 
with  a  tasking  to  produce  a  decision  brief  on  what  the  JSIMS 
was  to  constitute,  the  proposed  charter,  and  master  plan  in 
four  months.  Based  on  this  initial  timeline,  the  program 
could  be  viewed  as  approaching  a  year  behind  schedule. 

Since  the  JSIMS  MOA  was  signed,  the  project  has  moved 
from  a  general  consensus  agreement,  through  stand-up  of  a 
transitional  JSIMS  Joint  Program  Office  (JPO) ,  to  the 
formation  of  a  permanent  JPO.  The  Services,  Joint  Staff 
representatives,  and  the  Defense  Modeling  and  Simulation 
Office  (DMSO)  developed  a  JSIMS  system  definition  in  an 
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April  5-7,  1995,  meeting.  This  is  referenced  in  the  JSIMS 
Operational  Requirements  Document  (ORD)  and  was  used  in  the 
Milestone  0  decision  process.  The  ORD  has  been  through 
multiple  reviews  during  the  first  year  and  is  now  in  final 
coordination.  On  July  13,  1995,  the  Air  Force  Acquisition 
Executive  declared  that  JSIMS  had  attained  Milestone  0  and 
was  officially  in  the  Concept  Exploration  and  Definition 
phase  of  development.  On  July  14,  1995,  the  Director, 
Defense  Research  and  Engineering  (DDR&E) ,  chaired  the  first 
JSIMS  Senior  Review  Board  at  which  the  members  agreed  to 
provide  their  share  of  the  JSIMS  Joint  Program  Office  (JPO) 
permanent  staff. 

The  GAO  incorrectly  states  that  the  JSIMS  core  provides: 
common  functions,  like  weather  effects  and  terrain?  air, 
ground,  and  naval  combat  warfare  functions?  and  logistics. 
The  JSIMS  core  is  limited  to  shared  services,  interfaces, 
technical  infrastructure,  representations  of  the  Joint 
Community,  and  a  common  physical  environment.  The 
development  of  models  and  databases  for  representing  the 
physical  environment  will  be  provided  by  the  Under  Secretary 
of  Defense  for  Acquisition  and  Technology  (USp(A&T))  through 
designated  DoD  Modeling  and  Simulation  Executive  Agents 
(MSEA)  for  environmental  effects  representations.  The  MSEAs 
are  receiving  technical,  policy,  and  monetary  support  from 
the  DMSO  for  start-up.  The  Array,  Navy,  and  Air  Force  are 
taking  on  the  responsibility  to  be  JSIMS  Executive  Agents 
(JEAS)  for  their  primary  combat  area.  These  representa¬ 
tions,  while  part  of  JSIMS,  are  not  part  of  the  core.  Each 
JEA  is  developing  simulations  consistent  and  interoperable 
with  the  JSIMS  core  and  the  DoD  Modeling  and  Simulation 
common  technical  framework. 

The  GAO  is  correct  in  stating  that  the  Deputy  Secretary 
of  Defense  initiated  development  of  JWARS  to  obtained 
improved  DoD  analytical  capabilities.  The  GAO  incorrectly 
indicates  that  there  has  been  a  change  in  the  focus  of  ^ 
JSIMS?  its  primary  focus  has  always  been  on  joint  training 
with  analysis  as  a  secondary  capability  to  be  developed 
later.  Both  JWAR  and  JSIMS  will  comply  with  the  DoD  common 
technical  framework  being  developed  through  the  DoD  Modeling 
and  Simulation  Master  Plan,  which  will  support  their 
interoperability. 

The  Under  Secretary  of  Defense  for  Acquisition  and 
Technology  (USD{A&T)),  signed  an  action  memorandum  on 
August  8,  1995,  calling  on  DoD  Components  to  formally  adopt 
a  proposed  division  of  funding  and  personnel  requirements. 
The  Deputy  Secretary  of  Defense  endorsed  the  establishment 
of  a  joint  core  funding  line  with  the  Services  providing 
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both  their  share  of  core  funding  and  personnel  to  staff 
the  JPO.  The  Deputy  Secretary  of  Defense  endorsed  the 
establishment  of  a  joint  core  funding  line  with  the  Services 
providing  both  core  funding  and  personnel  to  staff  the  JPO. 
The  DDR&E  and  the  Joint  Staff  are  to  provide  a  portion  of 
the  funding  for  the  JSIMS  core  program.  The  Program  Element 
Monitor  (PEM)  is  assigned  to  the  Joint  Staff  J-7,  and 
program  development  is  managed  by  the  JSIMS  Joint  Program 
Office. 

The  GAO  incorrectly  reported  the  cost  to  develop 
JSIMS  through  initial  Operational  Capability  (IOC)  is 
$416  million.  In  fact,  this  figure  includes  not  only  the 
core  funding  ($154  million),  but  also  an  estimate  of  the 
funding  required  by  the  JSIMS  executive  agents  (land  -  Army, 
sea  -  Navy,  and  air  -  Air  Force)  to  develop  representations 
of  DoD  missions  in  their  respective  areas  ($262  million) . 

The  Department  cannot  substantiate  the  GAO's  $262  million 
estimate.  The  Army,  Navy,  and  Air  Force  are  now  developing, 
or  will  develop,  new  training  models  to  meet  their  Title  10, 
United  States  Code  training  needs.  It  is  probable  that  the 
Services  will  use  these  new  developments  as  contracting 
vehicles  to  meet  their  JEA  requirements  to  provide  the  JSIMS 
JPO  with  representations  of  their  respective  JEA  warfare 
areas.  The  Services  and  the  JPO  envision  significant  reuse 
of  the  functionality  and  technical  services  (Functional 
Description  of  the  Battlespace,  runtime  infrastructure, 
etc.)  between  and  among  the  Services'  new  training 
simulations  and  JSIMS.  As  a  result,  potential  exists  for 
savings  in  the  Service  Modeling  and  Simulation  programs  and 
the  JSIMS  total  program  cost. 

The  JEAs  will  not  have  to  fund  representations  included 
in  their  mission  areas  but  required  by  another  Service. 

For  example,  the  Army  would  either  be  provided  necessary 
representations  of  Marine  Corps  ground  combat  elements  or 
the  Marine  Corps  would  fund  the  production  of  their 
representations.  The  same  would  hold  true  for  the  Air 
Force  when  including  Navy  or  Army  aviation  elements. 
Additionally,  by  using  common  elements  provided  by  the 
JEAs  for  the  various  mission  areas  and  the  environmental 
representations  from  the  DoD  MSEA  denned  for  environmental 
representations,  all  may  experience  cost  avoidance  in  the 
long  term. 

Further,  the  DDR&E  and  the  Joint  Staff  are  to  provide  a 
share  of  funding  for  the  JSIMS  core  program.  Implementation 
will  be  reviewed  as  the  Department  finalizes  the  FY  1997 
budget . 
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Now  on  pp.  6‘7. 


PIIJDING  B:  The  BoD  Expenditures  On  The  Aggregate  Level 
gimulation  Protocol  fALSP)  Confederation  May  Not  Be  post- 
Effective.  The  GAO  reported  that  concurrent  with  the 
development  of  the  JSIMS,  the  DoD  has  decided  to  make 
improvements  to  the  ALSP  Confederation,  the  last  of  which 
is  expected  to  be  in  place  in  1999,  at  the  same  time  that 
the  JSIMS  should  reach  initial  operational  capability. 

The  GAO  stated  that  the  ALSP  Master  Plan  improvements  are 
intended  to  respond  to  the  identified  Commander  in  Chief 
(CINC)  and  Service  training  requirements  and  include 
additional  capabilities  such  as  strategic  mobility  and 
ground-to-ground  combat  interactions  between  models. 

Although  the  DoD  indicated  the  total  cost  of  these 
improvements  will  not  be  significantly  high,  the  GAO 
found  no  one  within  the  Office  of  the  Secretary  of  Defense 
and  the  Army  Simulation,  Training,  and  Instrumentation 
Command  was  able  to  provide  comprehensive  cost  estimates. 

The  GAO  identified  about  $40  million  that  the  DoD  has 
budgeted  for  the  ALSP  Confederation  improvements  through 
fiscal  year  1999.  The  GAO  concluded  that  because  this 
money  may  be  directed  toward  Service-specific  improvements, 
rather  than  joint  improvements,  the  cost  could  be  higher. 

The  GAO  reported  that  as  is  the  case  with  the  JSIMS,  there 
is  no  central  funding  line  for  the  ALSP  Confederation 
improvements.  Consequently,  the  GAO  concluded  that  the  DcpD 
ability  to  achieve  all  of  the  improvements  that  it  seeks  is 
dependent  on  funding  from  the  individual  Services,  agencies, 
or  the  CINCs.  The  GAO  noted  that  to  date  the  Office  of  the 
Under  Secretary  for  Acquisition  and  Technology  has  not 
provided  the  management  to  ensure  that  all  significant 
components  of  the  ALSP  improvements  will  be  completed.  The 
GAO  reported  that  management  improvements  have  been  frag¬ 
mented  and  it  is  questionable  whether  the  improvement  plan 
is  cost  effective.  For  example,  the  GAO  found  that  the 
Army  has  decided  not  to  fund  ALSP  improvements  to  its  ground 
warfare  model.  The  GAO  pointed  that  the  Army  will  make 
changes  for  Army  specific  traning  purposes  or  if  the  changes 
are  funded  by  a  Commander-in-Chief .  (pp.  2-3,  pp.  7-9/GAO 
Draft  Report) 

POD  RESPONSE;  Partially  concur.  Although  the  GAO 
identified  potential  inefficiencies,  the  draft  contains 
erroneous  budget  figures  and  incomplete  information 
concerning  the  management  of  the  ALSP  Core  Software. 

The  ALSP  software  provides  a  means  to  link,  simulations 
that  were  never  intended  to  interoperate,  and  it  provides 
the  only  current  method  linking  large  scale  heterogeneous 
simulations.  While  it  is  correct  that  ALSP  only  fully 
satisfies  2  of  the  25  Combatant  Command  training 
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requirements,  it  partially  satisfies  another  11,  Further, 
this  shortfall  was  a  principal  driver  for  the  development 
of  JSIMS.  The  GAO  report  states  that  about  $40  million  is 
budgeted  by  the  Department  of  Defense  for  ALSP  Confederation 
improvements  through  FY  1999.  The  Department,  however, 
cannot  substantiate  this  figure.  The  ALSP  Systems 
Engineering  Core  Support  for  FY  1996  is  currently  budgeted 
at  $1.42  million  for  FY  1996,  $1.49  million  for  FY  1997, 
$1.56  million  for  FY  1998,  and  $1.64  million  for  FY  1999. 

The  DoD  agrees  that  there  is  no  single  funding  line  for  ALSP 
system  upgrades.  There  is,  however,  a  single  funding  source 
for  ALSP  core  support  comprised  of  OSD,  JCS,  and  Service 
shares.  Funding  for  support  of  and  modifications  to  the 
ALSP  Infrastructure  Software  (AIS)  are  transferred  to  and 
administered  through  a  single  funding  line  at  the  Army's 
Simulation,  Training,  and  Instrumentation  Command  (STRICOM) . 
Since  the  ALSP  Management  Plan  was  approved  in  May  1993, 
a  means  to  address  the  whole  ALSP  change  process  from 
requirements  definition,  through  technical  analysis,  to 
cost  effectiveness  and  affordability  has  been  in  place. 
Decisions  concerning  prioritization  of  changes  to  the  AIS, 
protocols,  and  the  Joint  Training  Confederation  (JTC)  are 
made  by  an  ALSP  Review  Panel  which  includes  representatives 
of  the  funding  organizations.  The  Joint  Staff,  J-7,  Joint 
Warfighting  Center  (JWFC)  established  the  Joint  Training 
Group  to  identify  modeling  and  training  improvement 
initiatives  which  will  also  participate  in  the  ALSP 
improvement  process.  If  participants  in  the  requirements 
identification  and  prioritization  process  are  unable  to 
agree,  issues  are  brought  to  the  DoD  Executive  Council 
for  Modeling  and  Simulation  (EXCIMS)  by  the  ALSP 
Executive  Agent  for  resolution.  To  date,  this  has  not 
been  necessary.  Joint  training  requirements  are  the 
responsibility  of  the  Chairman  of  the  Joint  Chiefs  of 
Staff.  The  USD(A&T)  provides  oversight  of  the  Modeling 
and  Simulation  technical  means  to  support  the  training. 

The  requirements  submission  process  is  completely  open. 
Requirements,  are  submitted  through  the  Requirements  Review 
Panel  to  the  Review  Panel  for  approval.  Conflicts  are 
resolved  by  the  ALSP  Interface  Working  Group.  If  the 
interactions  require  a  change  to  the  AIS,  some  of  the 
$1.4  million  administered  through  the  ALSP  Executive 
Agent  may  be  applied  to  the  project.  All  other  funding 
for  modifications  to  the  affected  simulations  is  provided 
by  the  owning  or  requesting  DoD  Component. 

***** 
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Now  on  pp.  7-8. 


Now  on  pp.  7-8. 


RECOMMENDATIONS 


RECOMMENDATION  It  The  GAO  recommended  that  the  Secretary 
of  Defense  establish  a  joint  funding  line  for  the  core 
development  of  the  JSIMS  and  the  Secretaries  of  the  Army, 
Navy,  Air  Force  establish  funding  lines  for  their  respective 
executive  agent  JSIMS  responsibilities  regarding  warfare 
function  development,  (p.  lO/GAO  Draft  Report) 

POD  RESPONSE;  Concur.  The  DoD  has  already  taken  action  on 
this  matter.  The  DoD  is  in  the  process  of  establishing  a 
joint  funding  line  for  the  JSIMS  core  and  will  ensure 
Service  support  for  their  respective  combat  representations. 


RECOMMENDATION  2;  The  GAO  recommended  that  the  Secretary 
of  Defense  reguire  the  Under  Secretary  for  Acguisition  and 
Technology  to  assume  a  stronger  management  role  by 

defining  the  JSIMS  and  developing  a  definitive 
plan  of  action;  and 

•developing  a  transition  strategy  to  phase  out  the 
ALSP  and  phase  in  the  JSIMS.  This  strategy  should  be 
based  upon  cost  estimates  associated  with  modifying, 
expanding,  and  testing  the  ALSP  Confederation  to  decide 
which  improvements  to  the  ALSP  Confederation  provide 
benefits  that  are  cost-effective.  (p.  lO/GAO  Draft 
Report) 

DOD  RESPONSES  Partially  concur.  As  discussed  in  the 
Department  response  to  Finding  A,  JSIMS  is  progressing 
at  a  normal  programmatic  pace.  The  USD(A&T)  has  taken 
action  to  define  JSIMS  and  develop  a  definitive  plan 
of  action.  The  USD(A&T)  has  been  represented  at 
virtually  every  critical  decision  point  in  the  JSIMS 
program.  The  USD(A&T)  was  influential  in  developing 
the  management  structure  for  JSIMS  and  ensuring  that 
his  representatives  (DDR&E  and  DMSO)  were  involved. 

The  working  definition  of  JSIMS  was  crafted  by  Service 
representatives  and  representatives  from  DMSO,  JWFC, 
and  the  JPO  during  meetings  April  6-7,  1995.  On 
July  14,  1995,  the  Director,  Defense  Research  and 
Engineering  (DDR&E) ,  chaired  the  first  JSIMS  Senior 
Review  Board  at  which  the  members  agreed  to  provide 
their  share  of  the  JSIMS  Joint  Program  Office  (JPO) 
permanent  staff.  The  USD(A&T) ,  Modeling  and 
Simulation  signed  an  action  memorandum  on  August  8, 
1995,  calling  on  DoD  Components  to  formally  adopt  a 
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proposed  division  of  funding  and  personnel  require¬ 
ments.  The  JPO  continues  to  pursue  development 
activities  such  as  the  definition  of  the  JSIMS  Model 
of  the  Mission  Space,  leading  the  development  of  the 
DoD  Modeling  and  Simulation  High  Level  Architecture 
Proto-Federation  for  Operational  Level  Training  under 
DMSO  sponsorship,  and  in  close  liaison  with  the  JWARS 
program. 

A  plan  to  phase  out  ALSP  and  phase  in  JSIMS  will  be 
developed  based  on  both  technical  considerations 
provided  by  the  USD(A&T)  and  operational  considerations 
provided  by  the  Services  and  CINCS. 


(703092) 
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